\f0\b0\i0\outl0\shad0\fs48 \cf0 \expan0 \ul0 \line Introduction to System Logs \line and Auditing\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Table of Contents:\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line - Introduction\line - Shell History\line - SULOG\line - SYSLOG.CONF\line - SYSLOGD\line - UTMP\line - WTMP\line - LASTLOG\line - CRON\line - XFERLOG\line - LOGINLOG\line - Closing Notes\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Introduction - And it begins...\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 I wrote this paper to introduce some of the logs and audit methods used to monitor systems, and more importantly, check for intruders. A lot of such methods of detection have been replaced by the syslogd daemon, due to it's powerful monitoring capabilities. I generally didn't aim this paper towards people just starting out on UNIX, but it doesn't get very advanced ;-). I've tried to describe the more important files, the ones which a hacker should take notice of, and learn to work with. Once you get an idea how they process, you can understand what you have to edit- and how you can keep from being logged all together.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Shell History - $HOME ; $HOME/tmp\line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line Whenever you logout or exit from some shells, the contents of your command history are saved in a file. When the user relogs in, the file is fed to the history list, allowing the use of stored commands from the last session. This file's presence on the system mainly depends upon how it is set up.\line \line The number of commands stored by history is indicated in the environment variable HISTSIZE. For example, a <HISTSIZE> of 50 would keep a history file of the fifty most recent commands issued:\line \line i.e. HISTSIZE=50;export HISTSIZE\line \line The bourne shell <sh>, uses ".sh_history" , along with it's enhanced version <jsh> and the korn shell <ksh>. The Bourne-Again Shell <bash> uses ".bash_history", and the C shell <csh>, Z Shell <zsh>, and the Enhanced C Shell <tcsh>, use ".history". Their can also be backups of these files- "dead.letter", "*.bak", and "*~". \line \line The shell history isn't necessarily for auditing purposes, but can be very dangerous. This is one of the best ways of removing your shell history. (Only works under the csh and variations)\line \line $ mv .logout save\line $ echo rm .history>.logout \line $ echo rm .logout>>.logout \line $ echo mv save .logout>>.logout\line \line Also, try editing your shell profile to rename the history file (with the environment variable HISTFILE) - alot of admins won't notice what you've done, and go right by your history.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 SULOG - /var/adm/sulog <specified in /etc/default/su>\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line Sulogs record the program [su]s usage. It isn't usually very important, unless you have to login with a hacked account or brute-force from within the system. "/var/adm/sulog" is the default location at which the log resides, but can be changed, which is indicated in "/etc/default/su". A normal sulog entry is constructed as follows: \line \line SU date time result terminal user-newuser\line i.e - SU 02/24 20:29 + console logik-root\line \line The "+" or "-" sign indicates the su commands result. A "+" having returned a successful exit status, a "-" displaying just the opposite. This entry shows that on February 2, at 8:29, a local user <logik> successfully switched to the root account.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 SYSLOG.CONF - /etc/syslog.conf <or specified with option in syslogd>\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line "syslog.conf" is the configuration file for the syslogd system log daemon. It contains information used by syslogd to forward system messages to the appropriate log files and users. The syslog.conf looks like this:\line \line facility.severity <TAB> action (multiple entries can be setup using semi-colons)\line \line "facility" is where the name of a system facility which is producing the message goes, and "severity" is the importance of the message. Where "action" is is the action which will be performed when such a message (as specified in the facility.severity fields) occurs. The message can be sent to a file, to another host, to a group of users, or all users. Here's a sample entry:\line \line kern.crit /etc/kern.log\line \line This tells all messages generated by the kernel with a critical severity flag to be sent to a file called kern.log, in the /etc directory. Their are a lot of different options which can be specified, which I might get into in a different article. The syslogd configuration can be very helpful when covering your tracks, since it tells you where you can locate every syslogd handled record on the system.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 SYSLOGD - /usr/etc/syslogd ; /usr/sbin/syslogd\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line Syslogd processes system messages and sends them to log files and users, specified in it's configuration file <syslog.conf>. When the syslogd daemon starts up, it creates the file syslog.pid </etc/syslog.pid>, containing it's process ID.\line \line The syslogd daemon reads from the address family socket </dev/log>, from an internet address family socket specified in /etc/services, and for kernel messages, syslogd listens at /dev/klog.\line \line Syslogd depends mainly upon it's configuration file in the /etc directory <syslog.conf, which I've described above>, but can use an alternative configuration file, specified with an option. Here's an example log entry, which was configured to log all system daemon messages to "derr.log" in the /etc directory:\line \line Configuration entry:\line daemon.* /etc/derr.log\line \line Log entry:\line Feb 16 09:24:02 uco syslogd: restart\line Feb 16 09:28:12 uco rshd[844]: connect from logik@cs.xsec.org\line \line The syslog daemon is started on February 16, at 9:24 on the host uco. At 9:28, logik@cs.xsec.org rshs into uco, and is assigned a process ID of 844. The complexity of the syslogd daemon and the simplicity of setting up such powerful auditing is why it can be one of the most dangerous files to a hacker - that is, if they don't know how to handle it :-).\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 UTMP - /etc ; /var/adm ; /usr/adm ; /usr/var/adm ; /var/log\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line The utmp file records information about who is currently using the system. The file is made up of utmp structure entries, thus defined in the utmp.h file. It's setup like this:\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut line\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Contains the name of the terminal on which the user \line logged in.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut name\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Contains the name of the user who logged in.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut host\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Containing the name of the host from which the user \line remotely logged in, otherwise, a null string.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut time\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Containing the time at which the user logged in.\f2\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line Whenever a user logs in, the login program fills in the entry in utmp for the terminal on which the user logged in. When they log out, init clears that entry by setting ut name and ut host to null strings and ut time to the time at which the user logged out.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 WTMP - /etc ; /var/adm ; /usr/adm ; /usr/var/adm ; /var/log\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line The wtmp file records all logins and logouts. It also records shutdowns, reboots, and when the date command is used to change the system's date. The wtmp file has to be created manually, since the programs which manage it don't- so it may not appear on all systems, which turns record-keeping off. \line \line The login program writes entries for logins in wtmp identical to the messages placed in utmp. For logouts, init sets the following, and leaves ut name and ut host with a null value:\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut line\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 The terminal on which the user was logged in.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut time\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 The time at which the user logged off.\line \line When the system is shut down or restarted, the init program leaves ut host null, and sets the following:\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut line\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ~\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut name\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 shutdown or reboot \line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut time\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 The time at which the shutdown or reboot occurred.\line \line And finally, when the date is changed <ut host and name null>: \line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut line\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 |\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut time\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Previous system date.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut line\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 {\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ut time\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Changed system date.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 LASTLOG - /usr/var/adm ; /usr/adm ; /var/adm ; /var/log\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line The lastlog file records the most recent login-date for every user logged on. The file is made up of lastlog structure entries (similar to the utmp structure), which are defined in lastlog.h, and contain the following:\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ll time\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Contains the time at which the user logged in\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ll line\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Containing the name of the terminal on which the user \line logged in.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 ll host\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Containing the name of the host from which the user \line remotely logged\line in, otherwise, it's null.\line \line To retrieve the login date, the login program performs an lseek to a byte-offset in the lastlog, corresponding to the userid which is logging in. \line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 CRON - /var/spool/cron/ ; /var/cron/\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line Cron runs the crontab command, which allows a user to set up a list of tasks they want performed regularly, which is often used to automate the detection of intruders. The default directory for the crontabs is "/var/spool/cron/crontabs/". Here's a quick and easy way to check what a user has setup:\line \line crontab -l user \line i.e. crontab -l root\line \line Any output from the crontab is mailed to the owner of the tab, or to the user specified in the <MAILTO> environment variable in the crontab-which is one reason why it can be such a useful auditing tool.\line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 XFERLOG - /usr/adm/xferlog <or specified with option in ftpd>\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line The xferlog file contains records sent from the FTP server daemon, ftpd. Each entry looks like this:\line \line current-time transfer-time remote-host file-size filename transfer-type special-action-flag direction access-mode username service-name authentication-method authenticated-user-id \line \line This file can be very important, most so if you used ftpd to gain access to the host. \line Here's an example entry:\line \line \f1\b0\i0\outl0\shad0\fs20 \cf1 \expan0 \ul0 Fri Feb 12 11:24:46 1998 16 ecos.org 44618 /xkil.c a _ o r logik ftp 1 *\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line \line This entry shows that on Friday, February 12, 1998, at 11:24 (and 46 seconds), logik (an authenticated user who logged on from ecos.org) used the file transfer protocol to download a 44,618 byte ascii file named xkil.c from the root directory (which took 16 seconds to complete). He used RFC931 authentication, an authenticated user id was not available, and no special action was taken. <whew> \line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 LOGINLOG - /var/adm/loginlog ; /usr/adm/acct/loginlog\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line The loginlog records bad login attempts. Once five unsuccessful logins are attempted, the login program forwards a record of each bad login, and sleeps for 20 seconds. If loginlog doesn't exist, login drops the line, so no logging is done. Here's how a lastlog entry is setup:\line \line login name;tty specification;time\line \line Like WTMP, loginlog has to be created manually to function. Therefore, if loginlog doesn't exist, it doesn't log anything. Also, loginlog must be owned by root and in the sys group to operate. It's always good to remember that loginlog only records the bad logins after five consecutive unsuccessful attempts. \line \line \f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 Closing Notes - Yes, it finally ends.\f1\b0\i0\outl0\shad0\fs24 \cf1 \expan0 \ul0 \line Basically, many programs can talk to the syslogd daemon, which makes an exhaustive look at system auditing almost impossible. Also, I didn't really get into the less important logs and history files, accounting records, X-only mode auditing, and of course the hateable checksum checking software like hobgoblin and tripwire - not to mention much detail ;-). Maybe I'll write a more advanced article <with actual techniques on "correcting" the logs-hoho!>, if I get a good response to this one :-).\line \line Shout outs to van Hauser / THC, Mantis, Jabukie and everyone at Rootshell, Bugtraq, and Technotronic.\line \line Logik\line logik26@cris.com}
